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BACKGROUND OF THE INVENTION 

The present invention is related to a 
management method of a distributed computer system on a 
storage area network environment. 
5 A distributed computer system capable of 

improving the performance by distributing the 
processing into plural computers are widely utilized* 
Such a distributed computer system provides a client 
application with a service by way of a plurality of 

10 server appliances. 

Also, a computer used in such a computer 
system includes a plurality of ports for connecting the 
computer to a storage system, and the storage system 
includes a plurality of ports for connecting the 

15 storage system to a plurality of computers- A disk 
(storage device) employed in the storage system is 
connected via these ports to the computers* When the 
interconnections between the disk and the computers are 
concentrated into one port, the load exceeds the 

20 processing capability of this port. As a result, in 
the case that there are plural ports, the load balance 
of these ports are performed in such a manner that the 
interconnections between the disk and the computers are 
established via other ports. 

25 In a conventional distributed computer 



- 2 - 

system, the server appliance and the storage system are 
-connected to each other in the one-to-one 
correspondence. When the physical structure of the 
computer system is changed with the addition and 
5 deletion of the server appliance and the storage system 
in order to improve the maintenance of the server 
appliance, or to improve the processing performance of 
the entire computer system as well as to achieve the 
load balance of the entire computer system, the service 

10 provided by the computer system is completely stopped* 
Then, a system administrator changes the 
interconnection between the server appliance and the 
storage system, and also changes the system setting of 
the server appliance. Subsequently, the system 

15 administrator commences the service of this computer 
system. As a result, the system administrator is 
required to execute a large number of the system 
setting works. 

Similarly, when the port load balance is 

20 performed, the system administrator is required to 
reconfigure the computer system after the computer 
system is stopped. 

SUMMARY OF THE INVENTION 

The present invention provides a method of 
25 automatically and easily changing a server appliance 

for processing data in a distributed computer system by 
carrying out for a service provided by the computer 



system without completely stopping the computer system, 
the maintenance of the server appliance, the addition 
and deletion of the server appliance for the 
improvement of the system processing capability and the 
5 load balance, and the distribution of data into a 

plurality of disks for the disk load balance. Thereby, 
the reliability of the computer system can be improved, 
the work load of a system administrator can be reduced, 
and the mistake causing system down by the system 
10 administrator can be lowered. 

Also, the present invention provides means 
for changing a port used to interconnect a computer and 
a storage system without completely stopping a computer 
system. 

15 A network is constituted between a server 

appliance and a storage system to physically connect 
the server appliance and the storage system. A group 
of data to be processed by the computer system are 
partitioned. A plurality of server appliances for 

20 processing the partitioned data and a plurality of 
disks storing the partitioned data are connected via 
the network. When the load of one server appliance 
processing certain partitioned data is increased, the 
disk storing the partitioned data is notified to a 

25 destination server appliance in order that a portion of 
the partitioned data processed by the server appliance 
having the heavy load is processed by another server 
appliance. Thereby, the processing related to the 
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partitioned data is transferred. Here, the transfer of 
the processing related to the partitioned data is 
reflected on the setting of the service for providing 
the processing related to a group of data. 
5 Furthermore/ the partitioned data of certain 

group data is further partitioned. When the disk load 
balance is carried out by subdividing the further- 
partitioned data into a plurality of disks, a disk 
storing the further-partitioned data is notified to a 

10 server appliance to execute the processing of the 
further-partitioned data. Here, the fact that the 
partitioned data is further partitioned and the fact 
that the server appliance processing the partitioned 
data is allocated are reflected on the setting of the 

15 service for providing the processing related to the 
certain group data. 

Also, the present invention provides a 
graphical human interface capable of changing the 
system structure for deleting and/or adding a server 

20 appliance for processing data from the services by 
representing a plurality of server appliances for 
providing the services to a system administrator by a 
symbol to move the symbol on a screen. 

BRIEF DESCRIPTION OF THE DRAWINGS 
25 For the better understanding of the present 

invention, reference is made of the detailed 
description to be read in conjunction with the 



> 
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accompanying drawings, in which: 

Fig. 1 is a structural diagram of a computer 
system to which the present invention is applied; 

Fig. 2 is a schematic block diagram of an FES 

5 appliance; 

Fig. 3 represents FES system management 
information managed by the FES appliance; 

Fig. 4 is a schematic block diagram of a BES 

appliance; 

10 Fig. 5 is a schematic block diagram of a 

storage system; 

Fig. 6 is a schematic block diagram of a 
system management server program; 

Fig. 7 indicates a GUI provided by a system 
15 management server program; 

Fig. 8 shows server information managed by 
the system management server program; 

Fig. 9 represents disk information managed by 
the system management server program; 
20 Fig. 10 shows storage port information 

managed by the system management server program; 

Fig. 11 represents node-to-node connection 
information managed by the system management server 
program; 

25 Fig. 12 shows a processing flow of the BES 

deletion processing in the system management server 
program; 

Fig. 13 shows a processing flow of the BES 
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addition processing in the system management server 
program; 

Fig. 14 shows a processing flow of the data- 
processing BES change processing in the FES appliance; 
5 Fig. 15 shows a processing flow of the 

access-disk confirmation processing in the BES 
appliance; 

Fig. 16 shows a processing flow of the BES- 
load monitor processing in the system management server 
10 program; 

Fig. 17 shows a processing flow of the disk- 
load monitor processing in the system management server 
program; 

Fig. 18 shows a processing flow of the disk 
15 partition processing in the FES appliance; and 

Fig. 19 shows a processing flow of the port- 
load monitor processing in the system management server 
program. 



DESCRIPTION OF THE EMBODIMENTS 

20 (FIRST EMBODIMENT) 

Fig. 1 is a structural diagram of a computer 
system to which the present invention is applied. The 
computer system 1 includes a plurality of client 
computers 10a and 10b (totally referred to as a "client 

25 application 10" hereinafter) ; a plurality of server 
appliances 30 (namely, FES (Front End Server) 
appliances 100 and BES (Back End Server) appliances 
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200) ; storage systems 50a and 50b (totally referred to 
as a "storage system 50" hereinafter) connected via a 
Fiber Channel switch 40 to these server appliances 30; 
and a network 60. 
5 The storage system 50 includes disks 510, and 

storage ports 53 used to connect the storage system 50 
to the server appliances 30. The disk 510 in this 
embodiment is a logical device, and is arranged by a 
plurality of physical disks. However, since the disk 

10 510 has no direct relationship to the description of 
the embodiment, no explanation thereof is made. It 
should be noted that the disk 510 is not limited only 
to such a logical device. 

When the server appliance 30 connected to the 

15 storage port 53 is an open system such as UNIX, the 
storage port 53 may be an interface of SCSI (Small 
Computer System Interface) . Otherwise, when the server 
appliance 30 is a main frame, the storage port 53 may 
be a channel interface such as ESCON (Enterprise System 

20 CONnection) . Alternatively, the respective storage 
ports 53 may be made from the same interfaces and/or 
from different sorts of interfaces. In this 
embodiment, it is assumed that all of the interfaces 
are made from the Fiber Channel interfaces. 

25 The Fiber Channel switch 40 has a zoning 

function capable of limiting the connection between a 
server port 32 and the storage port 53 of the storage 
system 50. The Fiber Channel switch 40 may set the 



zoning by using a network interface 42. The server 
port 32 may access the storage port 53 within the same 
zone, whereas when the server port 32 and the storage 
port 53 belong to the different zones, no access 
5 operation can be done. The access operation may be 
limited by this zoning function. Although the zoning 
is not necessarily required, the zoning may prevent an 
unnecessary access operation from the server appliance 
30 to the storage system 50. The interconnection 

10 between the ports will imply that the ports are located 
within the same zone. 

The storage system 50 has a copy function 
capable of copying data of the disk 510 to another disk 
510. The copy function is described in U.S. Patent No. 

15 5,845,295, for example. 

Also, the storage system 50 applies an ID to 
the disk 510. The ID may guarantee that the disk 510 
is exclusively recognized at least within the computer 
system 1. For example, the storage ID is produced by 

20 using the vendor name of the storage system 50, the 
serial number of the storage system 50, and the 
internal identifier of the disk 510. 

The computer system 1 may provide at least 
one client computer with at least one service. A 

25 request issued from a client application with respect 
to a service provided in the computer system 1 is 
related to the processing for data having a certain 
implication. The service is provided by one FES 



appliance and at least one BES appliance. The data 
having the certain implication is divided into at least 
one to be stored in the disk. The FES appliance 
resolves the processing request issued from the client 
5 computer to the processing for the partitioned data. 

The partitioned data is stored in the disk, and the BES 
appliance which is instructed to access the disk and to 
process the partitioned data stored in this disk 
executes the processing, 

10 For example, there is a file server as such a 

service. The FES appliance may provide the client 
application with a directory structure (virtual 
directory) as such data having certain implication. 
The data is partitioned every a directory having a 

15 certain upper grade. The partitioned data is stored in 
the disk. A read request to the file contained in the 
virtual directory, which is issued from the client 
application, is resolved into the processing for the 
partitioned data which is partitioned from the path 

20 name to the file every directory. A processing request 
is sent to the BES appliance which is instructed to 
access the disk in which the partitioned data is stored 
and to process the partitioned data. The BES appliance 
receiving the processing request reads out the data 

25 from the disk, and then transfers the data of the 

designated field to the client application. There is a 
Web server or a database system as such a service, too. 

Fig. 2 is a schematic block diagram of the 



FES appliance 100. The FES appliance 100 includes a 
client-request decomposition unit 110, a data- 
processing BES changing unit 120, a disk partitioning 
unit 130, and a disk-load acquiring means 140,- and is 
5 constituted by FES system management information 150. 

The client-request decomposition unit 110 
decomposes the request issued from the client 
application into the processing for the partitioned 
data referring to the FES system management information 
10 150, and then allocates the decomposed processing to 
the BES appliance 200 which processes the partitioned 
data. 

The data-processing BES changing unit 120 
changes the method for allocating the BES appliance 

15 200, which processes the partitioned data, by the 

client-request decomposition unit 110. In other words, 
when the client-request decomposition unit 110 
allocates the partitioned data in such a manner that 
certain partitioned data is processed by a certain BES 

20 appliance, this certain partitioned data may be 

processed by another BES appliance different from the 
above-explained BES appliance by way of the change made 
by the data-processing BES changing unit 120. 

The disk partitioning unit 130 further 

25 partitions the partitioned data which the client- 
request decomposition unit 110 decomposes the request 
issued from the client application. In other words, 
when certain partitioned data is present, the data is 



further partitioned into two or more partitioned data, 
and then these further partitioned data are processed 
by the proper BES appliances* 

The disk-load acquiring means 140 acquires 
5 the disk load every disk which is acquired by the disk- 
load acquiring unit 230 of the BES appliance 200, and 
then updates the entry of the disk load of the FES 
system management information 150. The disk-load 
acquisition and the entry update operation are executed 

10 every time a certain constant time period has passed. 

The FES system management information 150 is 
the information of the service provided by the computer 
system, which is managed by each of the FES appliances. 
As shown in Fig, 3, this FES system management 

15 information 150 manages the partitioned data 

identifier, the disk ID of the disk in which the 
partitioned data is stored, the BES appliance which 
accesses the disk for storing the partitioned data in 
order to process the partitioned data, and the disk 

20 load. Based upon this information, the client-request 
decomposition unit 110 decomposes the request issued 
from the client application, and also allocates the 
decomposed requests to the BES appliances. The data- 
processing BES changing unit 120 changes this 

25 information, and changes the BES appliance processing 
certain partitioned data. When the partitioned data is 
further partitioned, the entry is additionally provided 
with this information. 
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Fig. 4 is a schematic block diagram of the 
BES appliance 200. The BES appliance 200 includes a 
data processing unit 210, a disk accessing unit 220, a 
disk-load acquiring unit 230, and an access-disk 
5 confirming unit 240 . 

The data processing unit 210 processes data 
stored in a disk which can be accessed by the disk 
accessing unit 220. Concretely speaking, the data 
processing unit 210 executes the processing in which 
10 the data stored in the disk is read to provide a file 
in accordance with a protocol; the database processing 
in which the data stored in the disk is retrieved/ 
updated; and another processing in which the data 
stored in the disk is read to provide data in 
15 accordance with a protocol. 

The disk accessing unit 220 accesses the disk 
of the storage system 50. 

The disk-load acquiring unit 230 acquires the 
load of the disk accessed by the disk access unit 220 
20 of the BES appliance 200. For instance, the disk-load 
acquiring unit 230 acquires the number of I/O during a 
certain constant time duration, acquires the percent 
with respect to the maximum I/O number, at which the 
disk can process, and provides the disk-load acquiring 
25 unit 140 of the FES appliance 100 with the disk load 
information. 

The access-disk confirming unit 240 checks 
the disk ID, which is exclusively guaranteed by the 
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system, to confirm whether or not a certain disk may be 
accessed by the disk accessing means 220, 

Fig. 5 is a schematic block diagram of the 
storage system 50. The storage system 50 includes a 
5 disk 510 corresponding to the data storage means, a 

data copy unit 550, and a port mapping change unit 560. 

The disk 510 is accessed by the disk 
accessing unit 220 of the BES appliance 200, and is 
exclusively identified by the disk ID which is 

10 exclusively used at least within the computer system 1. 
Although only one disk 510 is illustrated in Fig. 5, a 
plurality of disks 510 are generally employed in the 
storage system 510. 

The data copy unit 550 copies the data stored 

15 in the disk 510 to another disk 510 different from the 
first-mentioned disk 510. The other disk 510 to which 
the data is copied need not be necessarily employed in 
the storage system 50 containing the copy-source disk 
510, but may be employed in another storage system. 

20 The storage system 50 includes a plurality of 

ports for accepting the access issued from the BES 
appliance 200 to the disk 510. The storage system 50 
may determine via which port among these plural ports 
the disk 510 is connected to the BES appliance 200, and 

25 may change the relevant ports. The port mapping change 
unit 560 performs this port change. 

Fig. 6 is a schematic block diagram of a 
system management server program 400. The system 



management server program 4 00 includes a GUI 410, an 
FES system management information acquiring unit 420, a 
BES load monitoring unit 430, a disk load monitoring 
unit 440, a proper load monitoring unit 450, a BES 
5 addition processing 460, a BES deletion processing 470, 
server information 480, disk information 485, storage 
port information 490, and port connection information 
495. 

The GUI 410 provides an interface for easily 
10 managing services such as a file server, a database 
server and a Web server, which are provided by the 
computer system 1. As shown in Fig. 7, the FES 
appliance, the BES appliance and the disk are expressed 
by symbols. When the BES appliance handles the disk, 
15 the disk is connected to the BES appliance by a line. 
Every time a service is provided by the system, a 
rectangular shape is prepared, and the FES appliance, 
the BES appliance and the disk, which provide the 
service, are indicated in the same rectangular shape. 
20 The BES appliance and the disk which are not used are 
indicated in a rectangular shape indicative of "non- 
use . " 

When the user moves the symbol of the BES 
appliance, which is contained within the rectangular 
25 shape indicative of the service, to the rectangular 
shape indicative of "non-use", the BES deletion 
processing 470 is executed. As a result, the 
partitioned data, which is moved to the "non-use" 
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rectangular shape and is handled by the BES appliance, 
is allocated to another BES appliance for providing the 
service. Thereby, it is possible to cut the BES 
appliance away from the service without stopping the 
5 service by performing the simple operation by the GUI 
410 when the server appliance is brought into the 
maintenance work. When the maintenance work of the 
server appliance is completed, the user returns the BES 
appliance, which exists in the "non-use" rectangular 
10 shape on the GUI 410, to the rectangular shape 
indicative of the service to execute the server 
addition process 460. Thereby, the user can return the 
entire performance of the service to the original 
performance . 

15 The FES system management information 

acquiring unit 420 acquires the FES system management 
information 140 of the FES appliance 100. 

The BES load monitoring unit 430 is means for 
acquiring the load of the BES appliance 200, and for 

2 0 updating the server information 480. When the overload 
of the BES appliance 200 is detected by the magnitude 
of the load exceeding a certain threshold value, the 
BES load monitoring unit 430 executes the data- 
processing BES changing processing 120 in order to 

25 distribute the server load. This processing is 

executed every time a certain constant time period has 
passed. 

The disk load monitoring unit 440 acquires 
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the FES system management information 140 of the FES 
appliance 100 by using the FES system management 
information acquiring unit 420. When the overload of 
the disk is detected by the magnitude of the disk load 
5 exceeding a certain threshold value, the disk load 
monitoring unit 440 executes the disk partition 
processing 130 in order to distribute the disk load. 
This processing is executed every time a certain 
constant time period has passed. 
10 The port load monitoring processing 450 is 

means for acquiring the load of the storage port from 
the storage system, and for updating the port load of 
the storage port information 490. When the overload of 
the port is detected by the magnitude of the load 
15 exceeding a certain threshold value, the port load 

monitoring process 450 changes the port of the storage 
system, via which the disk is connected to the server, 
in order to distribute the port load. This processing 
is carried out every time a certain constant time 
20 period has passed. 

The BES addition processing 460 is the 
processing which is executed when a system 
administrator performs the operation to add a BES 
appliance 200 to service provided by the computer 
25 system 1 on the GUI 410. The BES addition processing 
460 enables the disk access unit 230 of the new-added 
BES appliance 200 to access the disk for storing 
certain partitioned data. Thereby, it is possible to 



process the partitioned data of the service in the new- 
added BES appliance 200. 

The BES deletion processing 470 is the 
processing which is executed when a system 
administrator performs the operation to delete a BES 
appliance 2 00 from service provided by the computer 
system 1 on the GUI 410. The BES deletion processing 
470 enables the disk access unit 23 of another BES 
appliance 200 providing the same service to access the 
disk for storing the partitioned data which is handled 
by the BES appliance 200 to be deleted. Thereby, the 
processing related to the partitioned data which is 
handled by the BES appliance 200 to be deleted is 
transferred to another BES appliance. 

The server information 480 is related to the 
FES appliance 100 and the BES appliance 200, which are 
used in the computer system 1. The server information 
480 is constituted by two tables as shown in Fig. 8. 
One of these two tables is constructed of the BES name, 
the FES name to which the BES presently belongs, the 
list of the FES appliances to which the BES can belong, 
and the server load of the BES appliance. The other of 
these two tables is constructed of the FES name, and 
the system name of service provided by the FES 
appliance . 

The disk information 485 is the information 
related to the disk 510 provided in the computer system 
1. As shown in Fig. 9, the disk information 485 is a 
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table constituted by the disk ID, the present use 
condition, and the storage system in which the disk is 
included. The disk information 485 is used to check 
whether or not there is any non-used disk when the data 
5 partitioning into multiple disks is made. 

The storage port information 490 is the 
information related to the port of the storage system. 
As shown in Fig* 10, the storage port information 490 
is a table constituted by the port name of the storage 

10 port, the ID of the storage system, the number of 

disks, and the port load. The storage port name is the 
World Wide Name (abbreviated as a "WWN" hereinafter) of 
the port of the storage system. The storage system ID 
is the identifier of the storage system which is 

15 exclusively defined within the system. The number of 
disks is the number of the disks 510 which are 
connected via the above-explained storage port to BES 
appliances 200. The port load is the load of the 
above-explained storage port. 

20 The port connection information 4 95 is the 

information related to the port which is used for the 
connection between a BES appliance 200 and a disk 510. 
As shown in Fig. 11, the port connection information 
495 is a table which is made of the server name, the 

25 server port name, the storage port name, and the disk 
ID. The server name is the name of the BES appliance 
200 accessing the disk 510. The server port name is 
the WWN of the server port used in the above-explained 



access operation. The storage port name is the WWN of 
the port of the storage system 50 used in the above- 
described access operation. The storage system ID is 
the identifier of the above-described storage apparatus 
5 50. 

Next, the processing flow in this embodiment 
will be described with reference to flow charts. 

The BES deletion processing 470 executed when 
a symbol of a BES appliance is moved from a rectangular 
10 shape indicative of service to a non-use rectangular 
shape by the GUI 410, which constitutes the operation 
for deleting a BES appliance, will now be explained 
with reference to Fig. 12. 

At a step 6010, the FES appliance to which 
15 the BES appliance designated to be deleted belongs is 
specified from the server information 480, and then the 
FES system management information 150 is acquired from 
the specified FES appliance. 

At a step 6020, all of the partitioned data 
2 0 which are handled by the BES appliance designated to be 
deleted, and the disk ID of the disk for storing the 
partitioned data are acquired form the FES system 
management information 150. 

At a step 6030, a move-destination BES 
25 appliance which takes over the partitioned data is 

determined among the BES appliances belonging to the 
same FES appliance as the BES appliance designated to 
be deleted from the server information 480. 
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At a step 6040, when the move-destination BES 
appliance cannot be determined, a notification that the 
deletion cannot be done is made to a system 
administrator at a step 6100, and then this processing 
5 is accomplished. To the contrary, when the move- 
destination BES appliance can be determined, the 
processing is advanced to a step 6050. 

At the step 6050, the data-processing BES 
changing unit 120 of the FES appliance to which the 

10 move-destination BES appliance belongs is executed by 
designating the deletion-designated BES appliance as 
the move-source BES appliance, by designating the 
partitioned data handled by the move-source BES 
appliance, and by designating the move-destination BES 

15 appliance determined at the step 6030. 

At a step 60 60, when the BES appliance cannot 
be changed, a notification that the deletion cannot be 
done is made to a system administrator at the step 
6100, and then the processing is ended. To the 

20 contrary, when the BES appliance can be changed, the 
entry of the server information 480 of the BES 
appliance designated to be deleted is changed into the 
"non-use" at a step 6070. 

This processing may also be executed in the 

25 case that the data processing is taken over another BES 
appliance in order to continue the service provided by 
the system when a certain BES appliance is brought into 
a malfunction state. 
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Referring to Fig. 13, the BES addition 
processing 460 executed when a symbol of a BES 
appliance is moved from a "non-use" rectangular shape 
to another rectangular shape indicative of the service 
5 by the GUI 410, which constitutes the operation capable 
of adding a BES appliance will be described. 

At a step 7010, it is checked whether or not 
a new-added BES appliance can belong to the FES 
appliance of the added-destination service, based on 

10 the server information 420. When the new-added BES 
appliance cannot belong to the FES appliance, a 
notification of "not added" is made to a system 
administrator at a step 7100, and then the processing 
is ended. To the contrary, when the new-added BES 

15 appliance can belong to the FES appliance, the 
processing is advanced to a step 7020. 

At the step 7 020, the FES system management 
information 150 is acquired from the FES appliance of 
the added-destination. 

20 At a step 7030, a being moved partitioned 

data is determined which is allocated from the FES 
system management information 150 to the added BES 
appliance . 

At a step 7 040, when the being moved 
25 partitioned data cannot be determined, a notification 
of "not added" is issued to the system administrator at 
a step 7100, and then the processing is ended. To the 
contrary, when the being moved partitioned data can be 
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determined, the processing is advanced to a step 7050. 

At the step 7050, the data-processing BES 
changing unit 120 of the FES appliance is executed by 
designating the BES appliance handling the being moved 
5 partitioned data as a move-source BES appliance, and by 
designating the being moved partitioned data and the 
added BES appliance as a move-destination BES . 

At a step 7060, when the BES appliance cannot 
be changed, a notification of "not added" is issued to 
10 a system administrator at the step 7100, and then the 
processing is ended. To the contrary, when the BES 
appliance can be changed, the processing is advanced to 
a step 7070. 

At the step 7070, the FES name of the added 
15 BES entry in the server information 48 0 is changed into 
the FES appliance of the service to which the BES 
appliance is added. 

This processing may be carried out also in 
such a case that the overall performance of the service 
20 is improved by adding the BES appliance when the load 
of the entire service is increased, other than such a 
case that the BES appliance whose maintenance has been 
carried out is returned to the service when the 
maintenance operation is accomplished. 
25 Next, a description will now be made of the 

data-processing BES changing process 120 executed in 
the BES appliance 4 60 and the BES deletion processing 
470 with reference to Fig. 14. 
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At a step 2010, a service temporary stop is 
instructed to the client-request decomposition unit 110 
of the FES appliance. Thereby, the decomposition of 
the processing from the client computer and the 
5 allocation of the processing to the BES appliance are 
temporarily interrupted. 

At a step 2020, with respect to the move- 
destination BES appliance of the processing for 
partitioned data, the disk ID of the disk storing the 
10 partitioned data is transferred to instruct the 

confirmation of the disk access possibility. Upon 
receipt of this instruction, the BES appliance confirms 
as to whether or not the disk 510 having the designated 
disk ID can be accessed by way of the access-disk 
15 confirming unit 240. 

At the step 2030, when the move-destination 
BES appliance can access the disk storing the being 
moved partitioned data, the processing is advanced to a 
step 2040. To the contrary, when the move-destination 
20 BES appliance cannot access this disk, the processing 
is advanced to a step 2100. At the step 2100, a 
notification of "not changeable" is issued to system 
administrator, and then the processing is ended. 

At a step 2040, the completion of the data 
25 processing for the being moved partitioned data is 

instructed to the data, processing unit 210 of the move- 
source BES appliance. 

At a step 2050, the commencement of the data 



processing for the being moved partitioned data is 
instructed to the data processing unit 210 of the move- 
destination BES appliance. 

At a step 2 0 60, the move-source BES appliance 
of the being moved partitioned data entry in the FES 
system management information 150 is changed into the 
move-destination BES appliance. The subsequent 
processing related to the partitioned data is allocated 
to the move-destination BES appliance. 

At a step 2070, the commencement of the 
service is instructed to the client-request 
decomposition unit 110 of the FES appliance. 

This processing changes a BES appliance which 
processes certain partitioned data, and may also be 
carried out by the BES load monitoring unit 430 which 
manages a load of a BES appliance. Further, this 
processing may be carried out in the following 
operation. That is, the BES-side edge of the line for 
connecting a symbol indicative of a BES appliance to a 
non-use rectangular shape indicative of a disk is moved 
to another BES appliance on the GUI 410. 

Fig. 15 is a flow chart of the access disk 
confirmation processing 240 in the BES appliance 200. 
The access-disk confirmation process 240 accepts a disk 
ID used to identify a disk and confirms whether or not 
the disk having the disk ID can be accessed. 

At a step 4 010, when the disk 510 having the 
disk ID can be accessed, the processing is advanced to 



- 25 - 

a step 4020 at which a notification of "accessible" is 
issued* To the contrary, when the disk 510 cannot be 
accessed, the processing is advanced to a step 4100 at 
which "non-accessible" is issued. The confirmation of 
5 the disk 510 may be carried out in such a manner that 
the serial number obtained by issuing an "Inquiry" SCSI 
command is compared with the disk ID. Alternatively, 
this disk confirmation may be carried out by writing 
the disk ID into the specific area of the disk and by 

10 reading this written disk ID. 

Referring now to Fig. 16, the BES load 
monitor processing 430 will be explained. In the BES 
load monitor processing 430, when a server appliance 
having a heavy workload is found out by monitoring 

15 loads of server appliances, the BES load monitor 

processing 430 changes system configuration so that the 
data, which is processed by the server appliance having 
a heavy workload, is processed by another server 
appliance to distribute the load. This processing is 

20 carried out in the system management server 400 in a 
periodic manner. 

At a step 5010, the system management server 
400 interrogates loads of BES appliances having the 
entry in the server information 480. 

25 At a step 5020, the server loads of the 

server information 48 0 are updated. 

At a step 5030, when there is no overloaded 
server, the processing is ended. To the contrary, when 



an overloaded server is detected, the processing is 
advanced to a step 5100. 

At this step 5100, the FES appliance, to 
which the overloaded BES appliance belongs, is acquired 
from the server information 4 80. The FES system 
management information 150 is acquired from the FES 
appliance. The entries, in which the overloaded BES 
appliance becomes the data processing BES appliance, 
are derived. Then, the data (disk) to be moved, this 
means a BES appliance processing the data will be 
changed, is determined among these entries. As the 
determining method, for instance, the partitioned data 
having the largest disk load may be determined to be 
moved, which means to be processed by another server 
appliance. 

At a step 5110, BES appliances - belonging to 
the same FES appliance as the overloaded BES appliance 
are derived from the server information 480. Then, a 
BES appliance which becomes the data destination is 
determined from the retrieved BES appliances. As the 
BES appliance determining method, for example, a BES 
appliance having a smallest load may be determined as a 
destination BES appliance. 

At a step 5120, when data to be moved and a 
destination BES appliance cannot be determined, the 
processing is advanced to a step 5130. When any one of 
the data to be moved and a destination BES appliance 
cannot be determined, the processing is advanced to a 



step 5200. 

At this step 5130, the data-processing BES 
changing process 120 is carried out by designating the 
BES appliance, in which the overload is detected, as a 
source BES appliance, and by designating the moved data 
determined at the step 5100 and the destination BES 
determined at the step 5110. Thereby, a BES appliance 
processing the data is changed from the move-source BES 
appliance into the move-destination BES appliance, so 
that the load of the move-source BES appliance can be 
reduced. 

At a step 5140, when a BES appliance for 
processing the data can be changed, the processing is 
ended. To the contrary, when a BES appliance for 
processing the data cannot be changed, the processing 
is advanced to a step 5200. 

At the step 5200, a notification that the BES 
appliance is over-loaded and the disk cannot be moved 
is issued to a system administrator, and then the 
processing is ended. 

The above-explained BES load monitor 
processing 430 is the processing for distributing the 
loads of the server appliances. There is such a case 
that I/O of a disk 510 accessed by a server appliance 
becomes a bottleneck of the system. To avoid this, the 
disk load monitor processing 440 is carried out by the 
system management server 400 in a regular manner. 
Referring now to Fig. 17, the disk load monitor 
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processing 440 will be explained. 

At a step 8010, the FES system management 
information 150 is acquired from the FES appliance 
having the entry in the server information 150. 
5 At a step 8020, when there is an overloaded 

disk, the processing is advanced to a step 8030. To 
the contrary, when there is no overloaded disk, the 
processing is ended. In the determination of the 
overloaded disk, for instance, the disk load monitor 
10 processing 44 0 judges the overload when the disk I/O of 
the FES system management information 150 exceeds a 
certain threshold value. 

At a step 8030, the data partitioning on 
multiple disks of the overloaded disk is executed by 
15 the disk partitioning unit 130 of the FES appliance to 
which the overloaded disk belongs. 

At a step 8040, when the data partitioning on 
the multiple disks can be carried out, the processing 
is ended. To the contrary, when the data partitioning 
20 on multiple disks cannot be carried out, the processing 
is advanced to a step 8100. 

At the step 8100, when the data partitioning 
on multiple disks cannot be carried out, a notification 
that the disk is overloaded and the data partitioning 
25 on multiple disks cannot be carried out is issued to a 
system administrator. 

In this processing, the disk load monitor 
processing 440 monitors the load of the disk accessed 
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by the BES appliance executing the data processing. 
When a disk having the heavy load is found out, the 
data stored in this disk having the heavy load is 
further partitioned, and then the further-partitioned 
5 data are stored into other disks. The BES appliance 
can access the disk storing the further-partitioned 
data during this processing, so that the disk load 
monitor processing 440 may distribute the disk load 
without stopping the service provided by the system. 

10 Referring now to Fig, 18, a description will 

be made of the data partition processing on multiple 
disks, which is called from the disk load monitor 
processing 440, 

At a step 3010, the partitioning number of 

15 the data to be partitioned, and the partitioned data ID 
are determined. 

At a step 3020, it is confirmed that there 
are sufficient disks, the number of which is equal to 
the partitioning number, referring to the disk 

20 information 485, 

At a step 3030, when there are the disks the 
number of which is equal to the total partitioning 
number, the processing is advanced to a step 3040, To 
the contrary, when there is no such a disk, a 

25 notification of "not partitioned" is issued at a step 
3200. 

At a step 3040, the use condition of the 
entry corresponding to the disk information 485 secured 
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at the step 3 02 0 is changed into the "use." 

At a step 3050, the disk copy instruction is 
issued to the data copy unit 550 of the storage system 
50 to copy the data from the disk to be partitioned 
5 onto the disks secured at the step 3020. 

At a step 3060, the service temporary 
interrupt is instructed to the client-request 
decomposition unit 110 of the FFS appliance to which 
the BES appliance handling the data to be partitioned 
10 belongs. The BES appliance will be referred to the 
"objective BES appliance" in this processing. 

At a step 3070, the execution of the access 
disk confirmation processing 240 to the disk 510 
secured at the step 3020 is instructed to the BES 
15 appliance handling the data to be partitioned. 

At a step 3080, when the disk can be 
accessed, the processing is advanced to a step 3090 . 
To the contrary, when the disk cannot be accessed, the 
entry of the disk information 485 of the disk secured 
20 at the step 3020 is returned to "non-use" at a step 
3300. At the next step 3310, a notification of "not 
partitioned" is issued, and then the processing is 
advanced to a step 3140. 

At a step 3090, the end of the processing for 
25 the data to be partitioned is instructed to the 
objective BES appliance. 

At a step 3110, the entry of the data to be 
partitioned of the FES system management information 



150 of the FES appliance to which the objective BES 

appliance belongs is deleted. 

At a step 3120, the commencement of the data 

processing to the partitioned data is instructed to the 

objective BES appliance. 

At a step 3130, the entry is added to the FES 

system management information 150 of the FES appliance 

to which the objective BES appliance belongs. This 

entry is constructed of the partitioned data ID after 

the data partition operation, the disk ID, and the 

objective BES appliance. 

At a step 3140, the restart of the service is 

instructed to the client-request decomposition unit 110 

of the FES appliance to which the objective BES 

appliance belongs. 

This processing is the following processing. 

That is, certain partitioned data is further 
partitioned to at least two. The disk for storing the 

further partitioned data is prepared. It is confirmed 
that the BES appliance can access the above-described 
disk. Thereby, the system operation is continued. 
This processing may be carried out also when the disk 
storing certain partitioned data and the data partition 
is selected from the menu, by the GUI 410 in addition 
to the disk load monitoring unit 440. 

The load of the BES appliance is distributed 
by the BES load monitor processing 430, and the load of 
the disk is distributed by the disk load monitor 
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processing 440. However, since the BES 200 accesses to 
the disk 510 via the port, there are some cases that 
the processing performance of the port may constitute 
the bottleneck of the system. 
5 When there are plural ports in the storage 

system, the storage port load monitor processing 450 is 
employed to distribute the loads of the ports, which 
will now be explained with reference to Fig. 19. 

At a step 9010, I/O loads of storage ports 
10 are acquired from the storage system 50. 

At a step 9020, the port loads of the storage 
port information 490 are updated. 

At a step 9030, when an overloaded storage 
port is present, the processing is advanced to a step 
15 9040. To the contrary, when there is no such an 
overloaded storage port, the processing is ended. 

At the step 9040, when an overloaded storage 
port is present, the disk connected via the overloaded 
storage port is derived from the port interconnection 
20 information 495, and then the disk the storage port of 
which is changed is determined. This disk is called 
the port-changing disk. 

At a step 9050, referring to the storage port 
information 490, when there is a port whose load is not 
25 so high within the same storage system as the 

overloaded port, the processing is advanced to a step 
9060. To the contrary, when there is no port whose 
load is not so high within the same storage system as 
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the overloaded port, the processing is advanced to a 
step 9200. 

At the step 9200, when there is no port whose 
load is not so high within the same storage system as 
5 the overloaded port, it is determined based on the 
storage port information 490 and the disk information 
485 whether or not the port whose load is not so high 
and the non-used disk are present within another 
storage system. When such a determination cannot be 

10 made, the processing is ended. To the contrary, when 
such a determination can be made, the processing is 
advanced to a step 9210. 

At the step 9210, the entry of the disk 
information 4 85 corresponding to the disk secured at 

15 the step 9200 is changed into "use." 

At a step 9220, the data copy is instructed 
to the storage system, which copies the data from the 
port-changing source disk to the port-changing 
destination disk . 

20 At a step 9060, the BES appliance using the 

port-changing (source) disk is derived from the port 
interconnect information 495. After the FES appliance 
to which this BES appliance belongs is acquired from 
the server information 480, the service temporary 

25 interrupt is instructed to the client-request 

decomposition unit 110 of the acquired FES appliance. 

At a step 9070, the completion of the data 
processing is instructed to the BES appliance handling 
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the port-changing (source) disk. This data processing 
is performed for the partitioned data stored in the 
disk whose port is not yet changed. 

At a step 9080, the port changing instruction 
5 is issued to the port mapping change unit 560 of the 
storage system. 

At a step 9090, the storage port information 
490 and the port interconnection information 4 95 are 
changed. 

10 At a step 9100, the confirmation of the data 

access possibility to the port-changing (source) disk 
is instructed to the BES appliance handling the port- 
changing (source) disk. 

At a step 9110, when such an access can be 
15 done, the processing is advanced to a step 9120. To 
the contrary, when such an access cannot be done, the 
processing is advanced to a step 9300. 

At a step 9120, to the BES appliance 
executing the access disk confirmation processing 240 
20 at the step 9100, the commencement of the processing of 
the data stored in the port-changing (destination) disk 
is instructed. 

At a step 9130, the restart of the service is 
instructed to the client-request decomposition unit 110 
25 of the same FES appliance as that of the step 9060. 

This processing is carried out in a periodic 
manner in the system management server 400. Also, in 
this processing, the port loads of the storage ports 
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are balanced. Alternatively, as explained in the 
storage port information 490, it is possible to 
distribute the load of the server port by managing the 
server port information. 
5 In the distributed computer system, it is 

possible to expand the computer system by the addition 
of the server appliance and the storage system without 
stopping the overall computer system. 

When the server appliance is brought into a 

10 malfunction, it is possible to continue the operation 
of the computer system by connecting the disk, which is 
connected to the server appliance under malfunction, to 
another server appliance. As a result, it is possible 
to improve the failure-proof characteristic of the 

15 computer system. 

In order to distribute the disk load, the 
system setting condition is automatically changed in 
such a manner that the data stored in the disk is 
partitioned into the plural data to process the 

20 partitioned data at the same time. Thereby, the system 
administrator can readily execute to distribute the 
disk load without stopping the computer system. 

In order to distribute the port load, when 
the port used to interconnect the server and the disk 

25 is changed, the access to the disk whose port is 

changed is confirmed, and at the same time, the setting 
condition is automatically changed. Thereby, the 
computer system can be easily changed, and it is 
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possible to avoid the system-down caused by the 
operation which is mistakenly performed by the system 
administrator . 

The port used to interconnect the server and 
5 the disk is automatically changed, so that the 

throughput between the server and the disk can be 
maintained. 



